Skip to content

perf(core): server-side response caching using the Web Cache API#2895

Closed
iuioiua wants to merge 5 commits into
freshframework:mainfrom
iuioiua:cache
Closed

perf(core): server-side response caching using the Web Cache API#2895
iuioiua wants to merge 5 commits into
freshframework:mainfrom
iuioiua:cache

Conversation

@iuioiua

@iuioiua iuioiua commented May 7, 2025

Copy link
Copy Markdown
Contributor

This PR adds the ability for apps to cache responses based on GET requests using the Cache API. In the following benchmark, this has resulted in a ~14x speed increase in responses. This server-side caching has been enabled on the www server as a test bench.

I've taken a fairly blasé approach to this functionality in that it doesn't fully consider edge cases, etc. Rather, its just able to be enabled or disabled. Finer control can be gained in proceeding PRs based on real use.

We would also want to document this functionality in the documentation, once merged.

Closes #8

// www/bench.ts
import "./telemetry.ts";
import { App, fsRoutes, staticFiles, trailingSlashes } from "fresh";
import { app as cachedApp } from "./main.ts";

export const uncachedApp = new App({ root: import.meta.url })
  .use(staticFiles())
  .use(trailingSlashes("never"));

await fsRoutes(uncachedApp, {
  loadIsland: (path) => import(`./islands/${path}`),
  loadRoute: (path) => import(`./routes/${path}`),
});

const uncachedHandler = await uncachedApp.handler();
const cachedHandler = await cachedApp.handler();

Deno.bench(
  "uncached handler",
  { group: "handler" },
  async () => {
    await uncachedHandler(
      new Request(
        "http://localhost:8000/docs/examples/using-fresh-canary-version",
      ),
    );
  },
);

Deno.bench("cached handler", { group: "handler" }, async () => {
  await cachedHandler(
    new Request(
      "http://localhost:8000/docs/examples/using-fresh-canary-version",
    ),
  );
});
    CPU | Apple M2
Runtime | Deno 2.3.1 (aarch64-apple-darwin)

file:///Users/asher/GitHub/fresh/www/bench.ts

benchmark          time/iter (avg)        iter/s      (min … max)           p75      p99     p995
------------------ ----------------------------- --------------------- --------------------------

group handler
uncached handler: GA4_MEASUREMENT_ID environment variable not set. Google Analytics reporting disabled.
uncached handler          814.2 µs         1,228 (629.1 µs …   4.9 ms) 790.4 µs   2.8 ms   3.0 ms
cached handler             57.2 µs        17,480 ( 49.5 µs …   1.9 ms)  58.9 µs  72.5 µs  77.0 µs

summary
  cached handler
    14.23x faster than uncached handler

@iuioiua iuioiua marked this pull request as ready for review May 7, 2025 01:48
@iuioiua iuioiua mentioned this pull request May 7, 2025
@iuioiua iuioiua changed the title feat(core): server-side response caching using the Web Cache API per(core): server-side response caching using the Web Cache API May 7, 2025

@marvinhagemeister marvinhagemeister left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caching responses this way only works for a very small subset of demo apps. A GET request in itself says nothing about the caching behaviour. The assumption that it always returns the same result is wrong. The only characteristic that's often associated with the GET method is that it promises to not modify data on the server side. There is no guarantee for that either though.

Example: You have a website with users where they can view their profiles.

  1. User goes to /profile, request is cached.
  2. User updates their profile name on a different page
  3. User navigates again to /profile, but because the earlier request was cached we'll get a stale user name

@csvn

csvn commented May 7, 2025

Copy link
Copy Markdown
Contributor

This is probably better implemented as a middleware? It should be possible to use the FreshContext and dynamically determine if a Response can be cached:

const app = new App({ root: import.meta.url });

app.use(cacheMiddleware({
  webCache: await caches.open('fresh'),

  // Determine whether a page can be cached
  include: (ctx) => {
    if (ctx.state.isStaticPage) return true;
    if (ctx.state.userId === undefined) return true;
    return false;
});

Does the Deno Cache API respect the Vary header? For simple sites that maybe only have a user_session cookie or similar, it might work to use Vary: Cookie, even though it seems that it's a bit discouraged.

const app = new App({ root: import.meta.url });

app.use(cacheMiddleware({
  webCache: await caches.open('fresh'),

  // A list of `Vary` headers, `true` or `false`
  cacheable: (ctx) => {
    if (ctx.state.isStaticPage) return true;
    if (ctx.state.userId === undefined) return ['Cookies'];
    return false;
});

@marvinhagemeister

Copy link
Copy Markdown
Contributor

+1 Agree, yeah it should be a middleware and it should respect the relevant caching headers.

@iuioiua iuioiua marked this pull request as draft May 8, 2025 00:41
@iuioiua iuioiua changed the title per(core): server-side response caching using the Web Cache API perf(core): server-side response caching using the Web Cache API May 14, 2025
@iuioiua

iuioiua commented May 22, 2025

Copy link
Copy Markdown
Contributor Author

Superseded by #2989

@iuioiua iuioiua closed this May 22, 2025
@iuioiua iuioiua deleted the cache branch May 22, 2025 02:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Caching of pages

3 participants